Re: Hardware timestamp ?

[Guy Harris <guy () alum mit edu>]

On Feb 26, 2009, at 4:49 PM, Pierre Karampournis wrote:

> I am currently working in a university lab and I need to capture  
> packets with a nanosecond precision timestamp using the Pcap format.

There's precision, and there's accuracy.  If, for example, the  
mechanism that delivers packets to libpcap can determine the current  
time, with nanosecond resolution, when it queues up the packet for  
libpcap to read, that's high precision - but low accuracy, if you're  
trying to measure the time at which the packet arrived at the host,  
because there's a code path of indeterminate length between the  
arrival of the last bit of the packet on the receiving network adapter  
and the delivery of the packet to the mechanism, as well as other  
delays due to interrupt latency, interrupt batching, scheduling, etc.,  
etc..  (Yes, scheduling - you might, for example, have an interrupt  
handler that does relatively little work and that wakes up a thread  
that actually reads the packets from the adapter's buffer ring, and  
that thread might be preemptable even in the kernel.)

> Pcap format allows nanosecond timestamping (even if it is not  
> implemented in libpcap yet) so I tried to modify libPcap and Tcpdump  
> to realize that. But my first naive and ugly modification was  
> erasing the kernel timestamp (currently in microseconds) by a user  
> timestamp (in nanosecond of course) so I loose precision trying to  
> gain precision...

Or, rather, lose accuracy, as, on top of all the inaccuracy of the  
kernel time stamp, you add the code path for user-mode time stamping  
and scheduling/preemption issues in userland.

> Thinking about that, user timestamping is imprecise due to  
> scheduling and other factors, but kernel timestamping, even if it is  
> a lot better, isn't really accurate too.

Yes.

> So I read few things about using hardware timestamping and I wonder  
> if something like that is being developed in libPcap or if solutions  
> exists ?

Libpcap isn't sufficient.  What you'd *really* need would be something  
such as a timer in the network adapter, so that the network adapter  
puts in the buffer ring the time stamp for the packet, as well as the  
packet data, plus a driver and capture software stack that passes that  
time stamp up to libpcap.

Endace's DAG cards:

        http://www.endace.com/dag-network-monitoring-cards.html

support that in hardware, and it looks as if the time stamp from the  
card is what's supplied through libpcap.  It currently doesn't support  
nanosecond resolution, but that's because libpcap doesn't.  Their  
cards are supported through pcap-dag.c on Linux and FreeBSD; you could  
modify pcap-dag.c to provide higher-resolution time stamps.

Note, however, that DAG cards are capture-only; they are *not*  
designed to be or intended to be full-blown network adapters that you  
can plug into the networking stack on your OS.

> With current networks bandwidth, why the time precision still isn't  
> better ?

Precision, or accuracy?

The precision is limited by the OS mechanisms used to supply packets  
to libpcap, as well as by libpcap itself.

The *accuracy* is limited by the fact that most network adapters  
aren't designed primarily for use when capturing traffic, so they  
don't do their own packet timestamping, and libpcap normally just  
plugs into the OS's built-in facilities for capturing packets, and  
those either use the OS's networking stack, which is designed  
primarily for regular network traffic rather than traffic capture and  
might sacrifice low latency for higher throughput (thus adding delays  
that get in the way of time stamping), and which plugs into device  
drivers *also* designed primarily for regular network traffic rather  
than traffic capture, or use something such as BPF that might bypass  
the networking stack but *doesn't* bypass the driver.

I.e., the lack of *accuracy* is arguably a feature for the system as a  
whole, as it comes, at least in part, from, for example, batching  
interrupts, so that in times of high traffic you might get one  
interrupt (or one clock tick for polling) for multiple packets.  For  
high-accuracy time stamping, the right answer might be to use  
hardware, and software all the way down to the hardware, designed for  
traffic capture.
-
This is the tcpdump-workers list.
Visit https://cod.sandelman.ca/ to unsubscribe.