CERBERUS: High Speed Snort Alert File Browser

[Dragos Ruiu <dr () dursec com>]

Announcing CERBERUS 
(the aardvark a.k.a. the earth pig - with apologies to Dave Sim :-)
http://dragos.com/cerberus/
----

What is a Cerberus?
----
Ok, I got tired of waiting for MySQL and various HTTP front ends
to databases to run queries and update.

I found myself needing to consistently look for the needle in the haystack:
the interesting alerts amongst the many "noise" alerts and false positives
that my Snort IDSes generate. I have grown tired of delay using all of the 
other front ends so I decided to write my own.  I wrote this tool for *me* 
but you may also get some utility out of it. I use it to filter out hosts I do
not care about and common false alarms from alert files rapidly. I 
also use it to merge multiple files from multiple sensors together to do
corellation.

It removes the need to have a complicated database back end with 
all of the attendant maintenance while using Snort. This tool lets 
you browse Snort unified alert files visually on a terminal (It's 
best if you use a _wide_ xterm to see all the fields).  It also 
lets you quickly remove false positive alerts and noisy hosts 
from capture files using rapid single keystroke commands. 

By using the merge option you can merge together mutliple files and
remove duplicate events (like from multiple sensors or files). You can 
merge in "live" alert files from running Snorts to get a pseudo 
real time alert display. Cerberus will merge and filter the duplicate 
events if you reload the same file in over again.

I've been playing around with 32Mb - half million alert files and 
it just filtered a 500K alert file down to 449 interesting alerts with 
almost no wait time.  It eleiminates delay waiting for database queries 
to finish because you can use Cerberus interactively in real time. It 
loads >100K alert files in under one second on a humble p3 
750Mhz with pc100 ram and slow IDE disks, and half a million 
record alert files take only a few seconds to insert into the 
embedded database - I'm very pleased with the speed...

The catch:  you should use a machine with hefty memory 
because Cerberus keeps the alerts in its own embedded
alert database.  But the good news is that as well as being 
fast - the storage overhead is very light. So make sure you 
use an alert file rotation size smaller than at most half your 
main memory and you should be fine.

How do you use Cerberus?
----
This program digests the output of the Snort unified output 
plug in - which if you aren't using you should be, because 
it's the fastest and most efficient way of logging data from 
Snort. It not only is efficient in output to disk but it retains 
tagging and reference information. Unified format is vastly 
superior (thanks to Marty and the folks at Sourcefire) to pcap 
format and you can use Marty's and Andrew's barnyard 
utility to generate pcap files from it. Unified output also has
the added advantage that the files don't just grow infinitely 
but roll over after they grow to a predefined limit.  I recommend 
32MB files for ease and speed of management.

You enable output from snort suitable for digestion by 
Cerberus by adding (or uncommenting) this line in 
your snort.conf file:
      output alert_unified: snort.alert.

Then feed the alert files from your /var/log/snort directory 
to Cerberus along with the sid-msg.map file from your 
snort distribution using this syntax:

cerberus <filename> [/path/to/sid-msg.map] [outfile]

Use a wide terminal window to see all the fields. Browse 
and filter the alerts with Cerberus using the single key 
commands in the menus at the bottom of the screen. Use
cursor keys and PgUp and PgDown to navigate. Note 
that sort requests are cumulative and remove adjacent 
alert records works from the cursor down.

Where to get CERBERUS:
----
http://dragos.com/cerberus/

Curently precompiled versions have been built for OpenBSD, 
FreeBSD, Linux, and OSX.  Solaris and Win32 versions will 
arrive shortly when I get access to a Sparc machine to build it 
and tweak a few things for the Windows version. You can find 
all these at the url above along with checksums for the 
executables in my preferred RIPE-MD160 format (and md5, 
sum, and sha1). I'm also workign on some staticaly linked 
versions that I'll put up when I find the time.

What to do if you get library complaints at run time?
----
Well Cerberus uses very little besides malloc/free, (fs)printf,
fopen/fread, localtime/strftime, str(l/n)cpy/strcmp and curses.
If it complains about libraries, create a symlink in /usr/lib
or wherever you keep libs from the version it's asking for
to the version you do have.  It should work just fine.

But where is the source?
----
Well there is a lot of work that has gone into Cerberus, and 
no-one has been paying my bills for the last few months while 
I've devoted a substantial amount of effort into this project, 
so I've decided to try something new, and to release this 
as shareware.  This is not cynicism on my part about open-source, 
just an experiment to cover my costs and after my development 
cost/time has been covered I will likely consider releasing the 
source as I still fundamentally trust in open source as the best 
way to move projects forward and build secure code.

But the good news is that I am making unmodified distribution 
of the binaries available, and individual non-commercial use 
free, as well as allowing commercial entities a 14 day 
tire-kicking period. Beyond that I would like to suggest that 
donations be made per copy. (I take VISA, Mastercard and 
PayPal :-) This of course relies on the honor system - you 
get to decide if you get enough value from this alert browser 
that you wish to support its continuing development. And I'm 
going to be darwinistic about developing it. It currently 
supports most of what _I_ need from it and I will let the 
donations guide how much additional enhancement and 
support to give it. Send me e-mail at dr () dursec com 
if you are interested in the full licensed version.

As far as the safety of these binaries, well I've been working 
hard on optimizing this code and they are now more than 
3x smaller than the original versions at around 20-30K of 
code.  With code less = faster = better!

That is a small enough a file to be hand inspected and a 
few people like HD Moore have also examined the binaries 
verified nothing is suspicious - if you want someone 
else's opinion besides mine...  I would argue you are safer
with these than with most commercial software... 

The full product will also allow the writing back out of alert 
files after filtering, and some other fun options. If there is 
sufficient demand and interest I may also release the 
windowing GUI multi-probe/corporate version of this system 
I've been fiddling with, but I'll need a few committed 
backers before I sink in the development effort into finishing 
that because my development team and I (heh, me :-P :-) 
need to earn a living too.

I will continue to improve this product and will eventually put 
up a Cerberus site at dragos.com, and am I always open to 
improvement suggestions, bug reports and feature requests 
but reserve the right to prioritize based on contributions :-). 
I have a few items I still wish to add and complete fairly 
soon such as reverse sorting and some other tweaks that 
I want/need to personally use for my own applications - so 
do check back every now and then. I'll update version 
numbers on the filenames. 

cheers,
--dr

P.s. Yes the name does come from the comic book epic by Dave Sim... 
Don't mess with the Aardvark - he kicks ass.
 
-- 
dr () dursec com   pgp: http://dragos.com/dr-dursec.asc
"The question of whether computers can think is like the question
  of whether submarines can swim." --Edsger Wybe Dijkstra 1930-2002



-------------------------------------------------------
This sf.net email is sponsored by: Dice - The leading online job board
for high-tech professionals. Search and apply for tech jobs today!
http://seeker.dice.com/seeker.epl?rel_code1
_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://www.geocrawler.com/redir-sf.php3?list=snort-users