Re: pcap flow extraction, Net::Frame is your friend

[GomoR <fd () gomor org>]

You could try with this simple Perl program.

It will print the source and destination IPv4 addresses, and the source 
and destination TCP ports.

It should be able the analyze a 6GB file, but in case it fails, look 
at editcap (shipped with Ethereal/Wireshark). editcap will help you 
to split a large file into multiple smaller files.

--8<--
#!/usr/bin/perl
use strict; use warnings;

use Getopt::Std;
my %opts;
getopts('f:F:p:w:', \%opts);

my $oDump;

die("Usage: $0\n".
    "\n".
    "   -f  file to read\n".
    "   -F  pcap filter to use\n".
    "") unless $opts{f};

use Net::Frame::Dump::Offline;
use Net::Frame::Simple;

$oDump = Net::Frame::Dump::Offline->new(file => $opts{f});
$oDump->filter($opts{F}) if $opts{F};

$oDump->start;

my $count = 0;
while (my $h = $oDump->next) {
   my $f   = Net::Frame::Simple->newFromDump($h);
   my $len = length($h->{raw});
   my $ts  = $h->{timestamp};
   next unless $f->ref->{IPv4};
   next unless $f->ref->{TCP};
   my $source      = $f->ref->{IPv4}->src;
   my $destination = $f->ref->{IPv4}->dst;
   my $srcService  = $f->ref->{TCP}->src;
   my $dstService  = $f->ref->{TCP}->dst;
   print 'o Frame number: '.++$count." (length: $len, timestamp: $ts)\n";
   print "Source:      $source\n";
   print "Destination: $destination\n";
   print "Src service: $srcService\n";
   print "Dst service: $dstService\n";
}

END { $oDump && $oDump->isRunning && $oDump->stop }
--8<--

On Thu, Dec 06, 2007 at 06:35:42PM +1100, Ivan . wrote:
> Hi,
>
> Does anyone have any ideas for flow information extraction from a rather
> large pcap file, 6 gigs?
>
> I am after the standard stuff, source, destination, service.
>
> Ethereal/wireshark is a no go, as it won't process the file due to size,
> tcpflow is OK, but a little untidy.
>
> any suggestions are appreciated, preferably open source and also has anyone
> used "tcpdstat" for something like this?
>
>
> thanks
> Ivan

> _______________________________________________
> Full-Disclosure - We believe in it.
> Charter: http://lists.grok.org.uk/full-disclosure-charter.html
> Hosted and sponsored by Secunia - http://secunia.com/

-- 
  ^  ___  ___             http://www.GomoR.org/          <-+
  | / __ |__/               Research Engineer              |
  | \__/ |  \     ---[ zsh$ alias psed='perl -pe ' ]---    |
  +-->  Net::Frame <=> http://search.cpan.org/~gomor/  <---+

_______________________________________________
Full-Disclosure - We believe in it.
Charter: http://lists.grok.org.uk/full-disclosure-charter.html
Hosted and sponsored by Secunia - http://secunia.com/