Re: Proposed new pcap format

["Ronnie Sahlberg" <ronnie_sahlberg () ozemail com au>]

Hi list,

Im pretty new to this discussion (only joined so i could participate in this
discussion)

I have looked at http://www.tcpdump.org/pcap/pcap.html
and have some comments.


3.1  "this makes the parsing of the file slower but it permits to append
several capture dumps at the same file"
----------------------------------------------------------------------------
----------------------------------
Every capture file starts with one SHB anyway so why would having the length
of data following this SHB prevent
appending to the file?
The precense of a length field in the SHB would not prevent    "cat cap1.cap
cap2.cap >cap3.cap" from working.
However,  during live high speed captures it would be undesirable to seek()
the file back and forth everytime we
append to the file so one should allow a wildcard. Say that len==0 would
mean :  length unknown, parse block by block until you find the next SHB.

A capture application could then initially write length==0 when it first
writes out the SHB and then not seek() back and write the real length until
it closes the SHB.

I think one should add a 64bit length field to the SHB which can optionally
be 0 meaning : length of SHB is unknown which would then cause
the readed to fallback to read one block at a time until the next SHB is
found.


WHY?
Assume huge captures, +100GByte captures or larger. A sequential scan would
be very slow.
A length field in the SHB would thus be VERY useful when working with very
large captures.
A capture application could be coded so that it would generate one huge file
containing a sequence of SHB.
Say each SHB would be 100MByte is size, or 10Milion packets, ...

One could then write a small application that could do :
   iterate through all SHBs in the huge file (would be quick)
   read the first packet from each SHB before skipping to the next one and
extract the timestamp from it.
   print the list  SHB index : timestamp

Such an application would be able to VERY quickly scan through a HUGE file
and tell me that the event I am interested in probably occured somewhere in
SHB number 327 328 or 329.
I could then run a different application  "extractSHB enormouscapture.cap
327 328 329 >manageablecapture.cap"

Please consider adding a length field to the SHB.  Extracting data from 1GB
captures are already painful enough today and capture files are only getting
bigger and bigger.


3.1
user application opotion and operating system both have the same code.
user application should have code == 4.
also maybe consider adding code:5   timestamp when this SHB was created
making operations such as above even easier.



3.3
the packet block has a description of which interface the packet was
captured on.
It should also have a mandatory flag that describes whether we picked the
packet from Rx or Tx on the interface.
This should be a mandatory field in the header (1 bit) and not optional.

That would make it much much easier to handle the case when you capture from
a HUNT/ETTERCAP box or if you capture from a
linux router on the -i all     "interface" since it would be much easier
then to just filter "outgoing packets only.


2.1
is it necessary to allow nesting?  would it be useful or wouldnt it only add
complexity?



Application specific blocks:
----------------------------
It would also be nice to have a mechanism to store user/application specific
data in a capture file.
I would like to be able to store  etherealisms like  color-filters,
capture-filters, display-filters etc etc inside a capture file
I would later send to colleagues.
If certain ethereal preference settings were required in order to view a
packet/capture properly I would also like
to store the preferences inside the capture.
Maybe even a PNG picture of an interesting graph generated by the
application?
I am certain there are other kinds of metadata that users of other tools
would like to store in the file as well.

The application specific blocks should be flexible enough to store any sort
of data of the applications own choice.



-
This is the tcpdump-workers list.
Visit https://lists.sandelman.ca/ to unsubscribe.