Re: Packet Payload

[Peter Van Epp <vanepp () sfu ca>]

On Tue, Aug 29, 2006 at 09:31:38AM -0400, xelerated wrote:
> Im posrting this to the pen-test group, rather than firewall or IDS
> because it covers many areas.
>
> Id like to see what the pro's think about capturing and storing packet
> payloads from firewalls, ids, etc... everything rather than just
> loggin the incidents.
>
> Im trying to explain to my management how useful the payloads could be
> if we were ever to
> really need them, say from a forensics point of view.
> To give another example, one time I was seeing lots of firewall drops,
> I could tell what ports, src and dest. but no packet data. To everyone
> involved it looked like a worm trying to spread.
> Well in the end it wasnt, infact is was something that was nice to
> know about, but it was not hostile traffic. But if I had been able to
> see the payloads i could have seen the data request and known from the
> start what it was, or was not.
>
> What would be really great, is a whitepaper covering this, or enough
> info/facts that I could throw one together.
>
> thanks!
> Chris
>
> C|EH, CISSP
>
> ------------------------------------------------------------------------
> This List Sponsored by: Cenzic
>
> Need to secure your web apps?
> Cenzic Hailstorm finds vulnerabilities fast.
> Click the link to buy it, try it or download Hailstorm for FREE.
> http://www.cenzic.com/products_services/download_hailstorm.php
> ------------------------------------------------------------------------

        If you can afford it this is a good idea. You probably want to do a 
calculation on exactly how much disk space and the sustained throughput to 
disk that are required to do this (hint, both are quite large and expensive 
for anything much beyond a T1 type circuit :-)). Then you will be wanting to 
be considering the security and privacy implications of this (think clear text 
passwords, credit card numbers, other confidential data that you are now 
proposing to take liability for keeping safe and unexposed on a continuing 
basis ...).
        Now that thats all said (and with the note that the same issues apply
to argus) you may want to take a look at argus (http://www.qosient.com/argus)
which is an ip auditing tool which captures netflow like data (and can be set
to capture the first 256 bytes or so of every connections data field. 
        Some example numbers from our ~35 meg per second commodity link which
say that for some 226 gigs of traffic the reduced argus output data amounted
to about 1.6 gigs of output from the argus sensor (which is much easier and 
cheaper to store for long periods of time). A full capture of this (relatively 
slow) link would be 226 gigs per day and getting that data captured and stored 
on disk is likely to be quite expensive because you need a fast disk subsystem
and a fast (probably independent of the disk machine) capture machine that 
can buffer in memory to survive disk writes on the storage box (which will
cause network packet loss on the network interfaces).

        Sensor to collector bandwidth reduction is quite high. Take the last   
24 hours on our commodity link. Input data was 226 gigs:

Traffic Summary From: Sun Aug 20  5:58:58 2006 To: Mon Aug 21  5:59:00 2006

     226,872,311,907 Total     160,526,269,792 Out      66,346,042,115 In      

and if I use 70 megs as an average file size (it varies from 51 to 82 megs
hourly across the period) thats about 1.6 gigs of argus output for 226 gigs   
on the wire for around a 140 to 1 reduction in bandwith needed between the   
sensor and the collector. So a 10 meg link would do fine at a gig and a 100 meg
backhaul should handle a full 10 gig link (if not a channel bonded pair of 100s
        Finally some papers on how we and other people use argus. 

http://www.usenix.org/publications/login/2001-11/pdfs/epp.pdf
http://www.malmedal.net/Malmedal_Master_Thesis.pdf
http://www.internet2.edu/presentations/jtvancouver/20050720-Argus-VanEpp.pdf

Peter Van Epp / Operations and Technical Support 
Simon Fraser University, Burnaby, B.C. Canada

------------------------------------------------------------------------
This List Sponsored by: Cenzic

Need to secure your web apps?
Cenzic Hailstorm finds vulnerabilities fast.
Click the link to buy it, try it or download Hailstorm for FREE.
http://www.cenzic.com/products_services/download_hailstorm.php
------------------------------------------------------------------------