tcpdump mailing list archives
Re: Proposed new pcap format
From: Jefferson Ogata <Jefferson.Ogata () noaa gov>
Date: Wed, 14 Apr 2004 13:27:29 -0400
Ronnie Sahlberg wrote:
Given all the desirable options people are looking for in this, and the need for future growth, I think we should seriously consider an XML-based format. Besides making it easy, format-wise, to include many optional features and types of metadata, programs could also embed decoded frame and protocol information in appropriate elements, right within the capture file. <capture ...>Please no. All programs reading pcap files through the pcap library will know how to translate the capture file into a dissected list of packets.
Again the tautology. Programs only need the pcap library to read capture files because of the file format. Make the file format into XML and any program that supports XML can read capture files. You don't need to compile pcap on new platforms just to read capture files; you can read pcap in Java or Perl or Python or PHP without finding a language port of libpcap. You can have your web browser display decoded capture files using an XSL stylesheet, without writing any new code. You can filter packets in decoded captures with XPath using expressions like "//*[dport = 53]" or "//arp" or "//ip[src = '127.0.0.1']" or "//frame[dir = 'inbound']" or "//frame[timestamp >= 1373849233]".
At the very least it argues for tcpdump in protocol dissection mode, and tethereal, to have XML output formats. Having the native capture file format be XML also, however, would turn protocol dissection into XML filtering, which would mean you could do it on raw capture files or preprocessed capture files alike. If you have two different file formats, your tools can only work on one or the other.
If this is absolutely necessary it can be done really well by an external tool thant reads a pcap file and expands it 1000 times into an xml file. It does not have to be implemented inside pcap.
Typical expansion would be by a factor of about 1.5 for undecoded packets, not 1000. Expansion for decoded packets would be somewhere between what tcpdump -v and tethereal do, more like 5 to 20.
NO xml in the kernel where pcap lives.
Huh? BPF lives in the kernel, on some platforms. pcap, and its file format, live in userland.
Also, some people actually work with pretty large files containint 10's of milions of packets.
Indeed. I am one of them. So what? -- Jefferson Ogata <Jefferson.Ogata () noaa gov> NOAA Computer Incident Response Team (N-CIRT) <ncirt () noaa gov> - This is the tcpdump-workers list. Visit https://lists.sandelman.ca/ to unsubscribe.
Current thread:
- Re: Proposed new pcap format, (continued)
- Re: Proposed new pcap format Christian Kreibich (Apr 13)
- Re: Proposed new pcap format Jefferson Ogata (Apr 14)
- Re: Proposed new pcap format Christian Kreibich (Apr 14)
- Re: Proposed new pcap format Hannes Gredler (Apr 14)
- Libpcap question Jacky Buyck (Apr 15)
- Re: Libpcap question Guy Harris (Apr 16)
- RE : Libpcap question Jacky Buyck (Apr 18)
- Re: Proposed new pcap format Guy Harris (Apr 14)
- Re: Proposed new pcap format Fulvio Risso (Apr 14)
- Re: Proposed new pcap format Ronnie Sahlberg (Apr 14)
- Re: Proposed new pcap format Jefferson Ogata (Apr 14)
- Re: Proposed new pcap format Fulvio Risso (Apr 14)
- Re: Proposed new pcap format Guy Harris (Apr 14)
- Re: Proposed new pcap format Fulvio Risso (Apr 13)
- Re: Proposed new pcap format Michael Richardson (Apr 16)
- Re: Proposed new pcap format Ronnie Sahlberg (Apr 11)
- Re: Proposed new pcap format Loris Degioanni (Apr 13)
- Re: Proposed new pcap format Fulvio Risso (Apr 13)
- Re: Proposed new pcap format Hannes Gredler (Apr 14)
- Re: Proposed new pcap format Fulvio Risso (Apr 14)