Re: parsing very large tcpdump files

[Vern Paxson <vern () icir org>]

> 1. Filter out traffic to/from a specific IP address or range
> 2. Reconstruct all reconstructable sessions in an easy to parse way: emails, web sites visited (and content 
> uploaded/downloaded), voip, anything else imaginable.
> 3. Be able to search all of this data for keywords. 

Bro is well suited for doing this.  It has a number of relevant hooks -
tcpdump/pcap filtering (via the restrict_filters/capture_filters script
variables, or at the command line, or via the "discarder" interface when
the list is too big to do via a filter) for (1), demuxing of reassembled
streams into individual files (via the contents.bro script) and app-level
summaries for apps it knows about for (2), and app-level event handlers +
its signature engine (for apps it doesn't know about), for (3).

You can get it from bro-ids.org.  If you wind up using contents.bro, drop me
a line, as we recently fixed a bug that can cause problems when it generates
thousands of files (the current public release doesn't yet include this).

                Vern

--------------------------------------------------------------------------
Test Your IDS

Is your IDS deployed correctly?
Find out quickly and easily by testing it with real-world attacks from 
CORE IMPACT.
Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708 
to learn more.
--------------------------------------------------------------------------