Re: pcap_next() caplen is off by 14 bytes (L2 len)

["Aaron Turner" <synfinatic () gmail com>]

On 3/20/07, Guy Harris <guy () alum mit edu> wrote:
> Aaron Turner wrote:
>
> > That's an excellent question.  The original pcap file is over 3 years
> > old, and honestly I don't remember.  My guess is that the packets were
> > most likely captured using tcpdump using the default snaplen on a
> > RedHat Linux box since that was my main development environment at the
> > time.   I honestly don't remember if I was using the RH provided
> > libpcap (I know there have been problems in the past) or one manually
> > compiled from source, let alone which version of libpcap it might of
> > been.  Of course, I may of captured the packets using Ethereal.
>
> It was almost certainly captured on a Linux box - it's not the standard
> libpcap format, it's one of a number of alternate formats used by some
> Linux libpcaps (see wiretap/libpcap.c for a bit of discussion of that).
>
> The issue is that the snapshot length in the file header is, for some
> reason, 14 bytes shorter than the *actual* snapshot length of the
> capture.  Ethereal/Wireshark ignores that value, but libpcap doesn't -
> it allocates a buffer large enough for the specified value, and discards
> packet data past that value.
>
> This might have come up before.

Yes.  Actually, I think I found this issue a few months ago, but
failed to put 2 & 2 together.

> Libpcap *could* just grow its buffer and read the data - correcting the
> snapshot length *ex post facto*.  However, any code that trusts the
> snapshot length value it gets after opening the file (by calling
> pcap_snapshot()) would be in for a rude surprise, getting a packet
> bigger than the biggest it was told the file would have.

Yeah, that wouldn't be nice.

> Any program that *completely* trusts that value - e.g., by allocating a
> buffer that big, and blindly copying the full packet into it - would
> break if we changed that.  Programs that report errors on those packets,
> or programs that report an error and stop, would also be a bit disturbed.

Not sure I follow the above.  What value are you talking about here...
the file snaplen?  Not sure what errors you're talking about here.
People should always compare the packet caplen/len to determine if the
packet was truncated, not the file snaplen.

> (Ethereal/Wireshark allocates a maximum-sized buffer for packets,
> regardless of the snapshot length.)
>
> So I'm not sure what the right thing to do here is.  One thing to do
> might be to write a "fixer" program that reads the file with a fake
> snapshot length of 65535 and writes it out with that snapshot length -
> or, if the snapshot length is worth saving in the file (e.g., to
> indicate what the limit was), do a two-pass scan.

One fix that would work w/o breaking backwards compatibility is to
emulate Ethereal/Wireshark for pcap_open_offline().  Basically ignore
the header snaplen, allocate the max size buffer and have
pcap_snapshot() always return 65535 as well.  I suppose someone might
assume that if snaplen > len, then len == caplen, in which case some
software may break, but it would be an easy fix.

The other option, which wouldn't change exiting functionality would be
to add a pcap_override_snapshot() for offline pcaps.  The idea here is
someone like me who is processing the data of the pcap can override
the default snaplen so that pcap_next() will return the "correct"
caplen values.  Subsequent calls to pcap_snapshot() would return this
new value.  Honestly I would prefer the first option (one less test I
need to write for autoconf) but I'd be quite happy with this.

Fixer programs in general are a problem, because diagnosing this issue
is very difficult.  Unless you compare the data in Wireshark and
tcpdump, there's no indication that you're being lied to by libpcap
and there's no way using the existing libpcap API to find out either.
Hence you don't even know you should run this fixer program.  As a
case in point, I've been using this test.pcap file for 3 years for my
unit test cases and I never noticed until now that this was happening.
Oops.


--
Aaron Turner
http://synfin.net/
http://tcpreplay.synfin.net/ - Pcap editing & replay tools for Unix
-
This is the tcpdump-workers list.
Visit https://cod.sandelman.ca/ to unsubscribe.