Re: Proposed new pcap format

["Loris Degioanni" <loris () netgroup-serv polito it>]

Jefferson,

> Fulvio Risso wrote:
> > > [mailto:tcpdump-workers-owner () lists sandelman ca]On Behalf Of Stephen
> > > Donnelly
> > >
> > > Jefferson Ogata wrote:
> > > > Yes, fully fledged decoded captures would use a lot of extra
> > > disk, but a
> > > > raw no-frills capture could be recorded with maybe only 50% or
> > > so overhead.
> > >
> > > 50% extra space and 50% extra disk bandwidth cost? So my 250
> > > Megabyte per second
> > > pcap stream to disk becomes 375MB/s?
> >
> > No, more than 500 MB/s.
> > You have to trasform everything in ascii, so an 8bit value becomes a 2
bytes
> > ascii value.
>
> As I imagine you know, XML is not ASCII; it's Unicode.
>
> Raw packet data would typically be base64-encoded. This expands data by
33%;
> three octets become four. You don't have to write one octet as two.
>
> In any case, if you're trying to capture every packet off the wire, you
might
> not want to use the newer binary pcap format under discussion either. It's
> looking to impose some not insignificant overhead as well.

I don't agree. If you don't put optional metadata in the packet, you are
going to have (at least when you save packets) something that is not too
different from current libpcap format. If you use the Simple Packet Block,
you have a even lighter solution.

Loris

> Again, pay attention to the discussion; there are many optional features
being
> suggested for the pcap storage format. What prompted my remark was the
> discussion about which hash algorithms to include in the storage format,
what
> data gets hashed, and whether any particular algorithm is designated as a
> default. That's the kind of stuff that says, to me, that a binary file
format is
> going to grow out of itself pretty fast.


-
This is the tcpdump-workers list.
Visit https://lists.sandelman.ca/ to unsubscribe.